Reinforcement Learning for Dynamically Stable Legged Lo- comotion

The Problem: The limit cycle stability of walking robots with only two legs, and even quickly moving robots with four or more legs, cannot be described as transitions between a set of stable fixed points in the configuration space of the robot. In these cases, the stability of the limit cycle depends critically on the dynamics of the walking machine. Because of the complexity of these dynamics, the fact that many of these robots are underactuated[3], and the uncertainty involved with walking on an unmodelled surface, the problem of constructing a control algorithm to provide this limit cycle behavior remains an open and interesting research question.